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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments, see remarks pages, filed 7-1-04, with respect to the 
rejection(s) of claim(s) 44-46, 48-49, 53-54, 56, 77-82, 88-90 and 95-97 under 35 U.S.C 
102(e) have been fully considered and are persuasive. Therefore, the rejection has 
been withdrawn. However, upon further consideration, a new ground(s) of rejection is 
made in view of Hotta (US 6,898,318) and Krumm et al. (US 6,532,301). 

Claim Objections 

2. Claims 53-55, 81 and 94 are objected to because of the following informalities: 
after "at least to a degree," there should be a -to- added. Appropriate correction is 
required. 

Priority 

3. The request for receiving the benefit of earlier filing date based on the application 
09/375,453 now US Patent 6,757,428 under 35 U.S.C. 120 and 37 CFR 1 .78 has been 
entered however it fails to meet the requirements of 35 U.S.C 112, first paragraph, for 
the subject matter of claims 55, 64, 65, 67-76 because ordinary skilled person in the art 
does not recognize that the applicant was in possession of the subject matter 
specifically "determining a step size" of claim 55, the "determining one or more 
dominant color category" of claims 64 and 65, the "searching a proximal region to the 
color match candidate" of claims 67-76. Therefore, claims 55, 64, 65, 67-76 are not 
entitled to the benefit of this applications and have an effective filing date no earlier than 
8/4/2000. 
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Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 88-96 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. Claim 88 is drawn to non-fucntional descriptive 
material. (See case law in MPEP 2106.IV.B.1(a)). Descriptive material that cannot 
exhibit any functional interrelationship with the way in which computing process are 
performed does not constitute a statutory process, machine, manufacture or 
composition of matter. Where a certain type of descriptive material, such as music, art, 
photographs and mere arrangements or compilations of facts or data, are merely stored 
so as to be read or outputted by the computer without creating any functional 
interrelationship, either as part of the stored data or as part of the computing process 
performed by the computer, then such descriptive material alone does not impart 
functionality either to the data as so structured, or to the computer. Claim 88 currently 

recites "A memory medium for locating ". There is no functional relationship imparted 

by this data to a computing device. Therefore, the claim is drawn to non-functional 
descriptive material which is non-statutory per se. The fact that claim recited a memory 
medium does not provide the utility required under 35 U.S.C 101 . The following formats 
are acceptable: "A computer program embodied in a computer readable medium for 
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performing the steps of ..." or "A computer readable medium storing a program for 
performing the steps of ... .". 

Claims 89-96 depend from a non-statutory claim and are thus themselves non- 
statutory. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 

form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

6. Claims 44-50, 53, 54, 77-81 , 88-91 , 94 and 95 are rejected under 35 
U.S.C. 102(e) as being anticipated by Hotta (US 6,898,318). 

Regarding claim 44, Hotta teaches, a method for locating regions of a target 
image that match a template image with respect to color characterization (Column 12, 
Lines 10-1 1 ), the method comprising: automatically determining color features of the 
template image (Column 12, Lines 11-14, wherein the color features of the template 
image is extracted and separated into RGB image data); locating one or more regions 
of the target image that match the color features of the template image (Column 12, 
Lines 14-22 wherein the closest correlated sub-images are detected and displayed 
Column 11, Lines 6-10); 
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Hotta teaches limitation of claim 45, the method further comprising: generating 
information specifying a location for each region of the target image that matches the 
color features of the template image (Column 11, Lines 6-10, wherein the location is 
marked with a line); 

Hotta teaches limitation of claim 46, the method further comprising: for at least 
one region of the target image that matches the color features of the template image, 
displaying information on a graphical user interface indicating a location of the region 
within the target image (Column 1 1 , Lines 6-1 1 ); 

Hotta teaches limitation of claim 47, the method further comprising: for at least 
one region of the target image that matches the color features of the template image, 
displaying information on a graphical user interface indicating a degree to which color 
information of the region matches color information of the template image (Column 1 1 , 
Lines 6-17, wherein the sub-image most closely correlated is selected of the plurality of 
displayed sub-images); 

Hotta teaches limitation of claim 48, the method further comprising: receiving the 
target image; wherein the target image is received from one of the group consisting of: 
a memory medium, a hardware device, and a software application (Column 12, Lines 
11-12, wherein the template and target image are taken from an external memory 
medium); 

Hotta teaches limitation of claim 49, either of the template image or the target 
image is a portion of a larger image (Column 7, Lines 8-10, wherein search image or 
target image of Figure 14 is portion of a larger image); 
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Hotta teaches limitation of claim 50, said locating one or more regions of the 
target image that match the color features of the template image comprises searching 
through the target image to find the one or more regions (Column 1 1 , Lines 53-56), the 
method further comprising: receiving user input specifying search criteria to use in 
searching through the target image; wherein the user input determines one or more 
parameters affecting said searching through the target image (Column 9, Lines 19-28, 
the user inputs a threshold value as correlation value); 

Hotta teaches limitation of claim 53, said automatically determining color features 
of the template image comprises performing a color characterization analysis of the 
template image (Column 12, Lines 10-14, wherein RGB data of the template image is 
the color characterization); wherein said locating one or more regions of the target 
image that match the color features of the template image comprises searching for 
regions of the target image having a color characterization that matches, at least to a 
degree, the color characterization of the template image (Column 12, Lines 14-22, 
wherein matching of template images with the search image is performed and some 
sub-images are detected and ultimately the ones have the most correlation detected as 
the optimal detection); 

Hotta teaches limitation of claim 54, said searching for regions of the target 
image comprises: performing a color characterization analysis for a plurality of regions 
within the target image to generate color characterization information for each of the 
target image regions; comparing the color characterization information of the template 
image with the color characterization information for each of the target image regions; 
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determining one or more target image regions having a color characterization that 
matches, at least to a degree, the color characterization of the template image (Column 
12, Lines 15-22); 

Hotta teaches limitation of claim 77, a system for locating regions of a target 
image that match a template image with respect to color and pattern information, the 
system comprising: a processor (Figure 17, item 1701); a memory medium coupled to 
the processor, wherein the memory medium stores color match location software 
(Figure 17, items 1703 and 1704); wherein the processor is operable to execute the 
color match location software to: automatically determine color features of the template 
image; locate one or more regions of the target image that match the color features of 
the template image (Figure 17, Item 1702, wherein combine units perform color 
matching of figure 16 method with their utility fully explained in Column ,12, Lines 30-45); 

Hotta teaches limitation of claim 78, the system further comprising: a display 
device; wherein, for each region of the target image that is determined to match the 
color features of the template image, the processor is operable to display information on 
a graphical user interface indicating a location of the region within the target image 
(Column 11, Lines 6-10); 

Hotta teaches limitation of claim 79, wherein the processor is operable to 
execute the color match location software to receive the target image from one of a 
memory medium and a hardware device (Column 12, Lines 11-12, wherein the template 
and target image are taken from an external memory medium); 
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Hotta teaches limitation of claim 80, wherein, in automatically determining color 
features of the template image, the processor is operable to perform a color 
characterization analysis of the template image (Column 12, Lines 10-14, wherein RGB 
data of the template image is the color characterization); 

Hotta teaches limitation of claim 81, wherein, in locating one or more regions of 
the target image that match the color features of the template image, the processor is 
operable to search for regions of the target image having a color characterization that 
matches, at least to a degree, the color characterization of the template image; wherein 
said searching for regions of the target image comprises performing a color 
characterization analysis for a plurality of regions within the target image (Column 12, 
Lines 14-22, wherein matching of template images with the search image is performed 
and some sub-images are detected and ultimately the ones have the most correlation 
detected as the optimal detection); 

Hotta teaches limitation of memory medium claims 88-91, 94 and 95 
corresponding to method claims 44, 46, 49, 50, 53 and 54, respectively, in column 30, 
Lines 36-38, wherein a utility of software in performing the processing of detected data 
of the method claims above is discussed. 

7. Claims 44, 51-54, 56-58, 66, 67, 77, 80-82, 86-88, 92-97 are rejected under 35 
U.S.C. 102(e) as being anticipated by Krumm et al. (US 6,532,301). 

Regarding claims 44 and 77, Krumm teaches, a method for locating regions of a 
target image that match a template image with respect to color characterization 
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(Column 2, Lines 27-46, also see figure 2, steps 200-210 and the system is outlined in 
Figure 1), the method comprising: automatically determining color features of the 
template image (Column 11, Lines 14-20, wherein pixel color characterization of 
template or model images are performed); locating one or more regions of the target 
image that match the color features of the template image (Column 13, Lines 51-57, 
wherein based upon the color characterization or color histogram, CH, the matching of 
the template or model images with search image is performed); 

Krumm teaches limitation of claims 51 and 86, said locating one or more regions 
of the target image that match the color features of the template image comprises 
performing multiple search passes through the target image according to a 
coarse-to-fine search heuristic (Column 14, Lines 25-28, where a subsequent higher 
resolution search is performed after a preliminary or coarse search) 

Krumm teaches limitation of claims 52 and 87, said locating one or more regions 
of the target image that match the color features of the template image comprises: 
performing a first-pass search through the target image to find initial match candidate 
areas (Column 13, Lines 51-57, wherein based upon the color characterization or color 
histogram, CH, the matching of the template or model images with search image is 
performed); performing one or more subsequent search passes in which proximal 
regions proximal to the candidate areas are searched in order to find a best-matching 
region in the proximal region (Column 14, Lines 25-40, wherein the proximal area of the 
identified regions or object is further searched); 
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Krumm teaches limitation of claims 53, 80 and 81 , said automatically determining 
color features of the template image comprises performing a color characterization 
analysis of the template image (Column 1 1 , Lines 14-20, wherein pixel color 
characterization of template or model images are performed); wherein said locating one 
or more regions of the target image that match the color features of the template image 
comprises searching for regions of the target image having a color characterization that 
matches, at least to a degree, the color characterization of the template image (Column 
13, Lines 51-63, wherein a preliminary threshold is set for comparison or matching of 
template regions to search image regions providing a bases for a matching degree); 

Krumm teaches limitation of claim 54, said searching for regions of the target 
image comprises: performing a color characterization analysis for a plurality of regions 
within the target image to generate color characterization information for each of the 
target image regions; comparing the color characterization information of the template 
image with the color characterization information for each of the target image regions; 
determining one or more target image regions having a color characterization that 
matches, at least to a degree, the color characterization of the template image (Figure 
2, Items 204 and 206 which effectively perform the same color characterization as 
discussed with respect to template or model images above and further Column 13, 
Lines 51-63 teaches the similarity or matching scheme of the regions); 

Krumm teaches limitation of claim 56 and 82, wherein the color characterization 
analysis performed for the template image and the color characterization analyses 
performed for each of the plurality of regions of the target image comprise: examining 



Application/Control Number: 09/639,420 Page 1 1 

Art Unit: 2621 

color information of at least a subset of pixels (Column 1 1 , Lines 1 8-20, wherein pixels 
are examined for their color); assigning each examined pixel to a color category that 
corresponds to a portion of a color space (Column 1 1 , Lines 20-21 , wherein each pixel 
is assigned to a bin by the histogram); determining information indicative of the 
allocation of the examined pixels across color categories (Column 1 1 , Lines 27-30, 
wherein the count or separating vector is the allocation information); wherein, for each 
of the plurality of regions of the target image, said searching comprises comparing the 
information obtained in the color characterization analysis of the region to the 
information obtained in the color characterization analysis of the template image in 
order to determine whether the region has a color characterization that matches, at 
least to a degree, the color characterization of the template image (Column 13, Lines 
51-57, wherein matching step is performed between the template and the search image 
within the requirement of the threshold set as the degree of matching inherently); 

Krumm teaches limitation of claim 57, the method further comprising: determining 
a sub-sampling size (Column 12, Lines 25-32, wherein a determination of subsampling 
of the search image is performed by dividing it to plurality of windows of a given size); 
wherein the sub-sampling size is used to determine the size of the at least a subset of 
pixels examined for each of the plurality of regions of the target image (Column 1 1 , 
Lines 33-39, wherein selection of the size of windows encompassing pixels within it for 
search is considered); 

Krumm teaches limitation of claim 58, wherein the color characterization analysis 
performed for the template image comprises examining color information of each pixel 
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in the template image; wherein the color characterization analyses performed for each 
of the plurality of regions of the target image comprise examining color information of 
only a subset of the pixels in the region (Column 14, Lines 45-52, wherein limited 
number of the pixels of the search image is compared with all of the model or template 
image pixel color information); 

Krumm teaches limitation of claim 66, the color characterization analysis 
performed for each of the plurality of regions of the target image further comprises 
performing a smoothing operation after said assigning, each examined pixel to a color 
category; wherein the smoothing operation comprises: for each respective color 
category of at least a subset of the possible color categories, re-distributing a portion of 
the pixels assigned to the respective color category to one or more neighboring color 
categories (Column 1 1 , Lines 39-50 wherein quantization of colors of pixels and its 
distribution to the most similar set is the smoothing operation); 

Krumm teaches limitation of claim 67, a computer-implemented method for 
locating regions of a target image that match a template image with respect to color 
characterization, wherein the target image and the template image each comprise color 
information, the method comprising: performing a first-pass search through the target 
image in order to find color match candidate areas (Column 13, Lines 51-57, wherein 
based upon the color characterization or color histogram, CH, the matching of the 
template or model images with search image is performed); for each color match 
candidate area found in the first-pass search, searching a proximal region proximal to 
the color match candidate area in order to find a best-matching region in the proximal 
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region (Column 14, Lines 25-40, wherein the proximal area of the identified regions or 
object is further searched); 

Krumm teaches limitation of claim 97, a computer-implemented method for 
locating regions of a target image that match a template image with respect to color 
characterization, wherein the target image and the template image each comprise a 
plurality of pixels, the method comprising: performing a color characterization analysis 
of the template image (Column 1 1 , Lines 16-20); and searching for regions of the target 
image having a color characterization that matches, at least to a degree, the color 
characterization of the template image (Column 13, Lines 51-63, wherein a preliminary 
threshold is set for comparison or matching of template regions to search image regions 
providing a bases for a matching degree); wherein said searching for regions of the 
target image comprises performing a color characterization analysis for a plurality of 
regions within the target image (Column 13, Lines 51-57, wherein matching step is 
performed between the template and the search image within the requirement of the 
threshold set as the degree of matching inherently); wherein the color characterization 
analysis performed for the template image and the color characterization analyses 
performed for each of the plurality of regions of the target image comprise: examining 
color information of at least a subset of pixels (Column 1 1 , Lines 18-20, wherein pixels 
are examined for their color); assigning each examined pixel to a color category that 
corresponds to a portion of a color space (Column 1 1 , Lines 20-21 , wherein each pixel 
is assigned to a bin by the histogram); determining information indicative of the 
allocation of the examined pixels across color categories (Column 1 1 , Lines 27-30, 
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wherein the count or separating vector is the allocation information); wherein, for each 
of the plurality of regions of the target image, said searching comprises comparing the 
information obtained in the color characterization analysis of the region to the 
information obtained in the color characterization analysis of the template image in 
order to determine whether the region has a color characterization that matches, at 
least to a degree, the color characterization of the template image (Column 13, Lines 
51-57, wherein matching step is performed between the template and the search image 
within the requirement of the threshold set as the degree of matching inherently); 

Krumm teaches limitation of system claims 77, 80-82, 86 and 87 corresponding 
to method claims 46, 51-53, 56 in Figure 1 ; 

Krumm teaches limitation of program instruction on a memory claims 88, 92-96 
corresponding to method claims 44, 51-54 and 56, respectively, in column 8, Lines 14- 
16. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 
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9. Claim 55 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Krumm 
(US 6,61 1 ,622) in view of Park et al (U.S. Patent No. 6,584,212). 

Regarding claim 55, Krumm teaches number of limitations of the claim (See 
previous office action with respect to rejection of claims 44, 53 and 54, dated 3-29-04), i. 
Determining a step size is not explicitly explained by Krumm. However, Park et al 
illustrate a first step size of two pixels in figure 1 by the two vectors originating from 
coordinate (0,0), ii. Determining locations for the plurality of regions within the target 
image for which the color characterization analysis is performed is illustrated by Krumm 
in figure 2 by reference numbers 200 and 202. Krumm explains determining the regions 
in column 9, line 53 to column 10, line 30, iii. The step size being used in determining 
locations for the plurality of regions within the target image is not explicitly explained by 
Krumm. However, Park et al explain in column 1 , lines 49-52 that a difference value is 
determined at every search location. Park et al explain in column 1, lines 11-18 that the 
multiple step search sizes reduce the calculations required to determine the best match 
in a target image. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to use a step size in order to determine locations 
for the plurality of regions within the target image used by the system of Krumm 
because the region selection process would be more efficient. 

10. Claim 62 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Krumm 
(US 6,532,301). 
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Krumm teaches limitation of claim 62, method further comprising: receiving user 
input specifying a desired color sensitivity level to use in locating target image regions 
that match the template image (Column 1 1 , Lines 36-39, wherein the quantization 
number is the level of sensitivity); wherein the user input determines a number of 
categories into which the color space is divided (Column 1 1 , Lines 39-42, where the 
category of colors are specified), in addition Krumm utilizes users input capability 
(Column 8, Lines 58-60) while the user input specifically regarding the above features is 
not discussed, the parameters required by the quantization step and the number of 
category of colors is inherently inputted by an operator for a given application. 
Therefore, absent some showing of criticality or unexpected results, the parameters 
required by the quantization step and the number of category of colors to be selected by 
an operator or user is within the skill level of ordinary practitioner in this art who would 
find it obvious to select the most appropriate parameters for a given application. 

1 1 . Claims 64-65 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Krumm (US 6,611,622) in view of Nelson et al (U.S. Patent No. 6,243,713). 

Referring to claim 64 (See previous action mailed 3-29-04 with respect to 
rejection of parent claims 44, 53, 54 and 56) determining one or more dominant color 
categories for the template image, wherein one or more dominant categories are 
assigned a relatively larger proportion of examined pixels, with respect to other 
categories of the color space is not explicitly explained by Krumm. However, Nelson et 
al explain in column 1 1 , lines 54-64 that only the top Nth most significant bins are 
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retained in the color histogram of an image. By determining the most dominant color 
categories for a template image, the amount of data being retained for each image is 
reduced, thereby reducing the computational power and memory required to store the 
characterization of the image. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to determine one or more dominant 
color categories for the template image, from the color histogram created in the system 
of Krumm because the computational power and memory requirement for the image 
matching system would be reduced; 

Referring to claim 65, for each dominant color category, comparing the 
percentage of template image pixels assigned to the dominant color category to the 
percentage of target image region pixels assigned to that color category is not explicitly 
explained by Krumm. However, Nelson et al explain determining the dominant color 
categories in column 1 1 , lines 54-64. Nelson et al also illustrate filtering and 
normalizing the tokens (i.e. categories) in figure 6 by reference numbers 630. The 
normalization of the categories determines the percentage of pixels present in each 
color category rather than the number of pixels. Nelson et al explain in the abstract that 
characterizations of multimedia (including image) works are created in order to retrieve 
multimedia from a database. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to compare the percentage of 
template image pixels assigned to the dominant color category to the percentage of 
target image region pixels assigned to that color category because comparing the 
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percentages of only the dominant categories reduces the computational power and 
memory requirement in the system of Krumm. 

12. Claims 67-70 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Krumm (6,61 1,622) in view of Hsu et al (U.S. Patent No. 6,078,701). 

Regarding claim 67, i. performing a first-pass search through the target image in 
order to find color match areas (Performing a first-pass search through the target image 
to find initial match candidate areas is not explicitly explained by Krumm. However, Hsu 
et al illustrate a coarse registration module in figure 3 by reference number 304; ii. 
Performing one or more subsequent search passes in which proximal regions proximal 
to the candidate areas are search in order to find a best-matching region in the proximal 
region is not explicitly explained by Krumm. However, Hsu et al illustrate a fine search 
in figure 3 by reference number 306. Hsu et al explain in column 5, lines 7-30 that the 
coarse and fine search steps validate hypothesis of neighbor matches. Hsu et al then 
explain that the search is conducted until a predefined level of topological and alignment 
accuracy is achieved in column 5, lines 31-42. Therefore, it would have been obvious 
to one of ordinary skill in the art at the time the invention was made to perform a first- 
pass and a second-pass through a target image to find accurate matching area because 
the multiple pass (or coarse-to-fine) search is well-known in the art to provide faster 
image matching, resulting in a more efficient color match system; 

Regarding claim 68, i. determining a plurality of sample regions at which to 
sample the color information of the target image is illustrated by Krumm in figure 2 by 
reference numbers 200 and 202, ii. for each of the plurality of sample regions; 
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determining a measure of difference between the color information of the sample region 
and the color information of the template image is illustrated by Krumm in figure 2 by 
reference number 204, iii. for each of the plurality of sample regions, designating the 
sample region as a color match candidate area if the measure of difference between the 
color information of the sample region and the color information of the template image is 
smaller than a threshold value is illustrated by Krumm in figure 2 by reference number 
206; 

Regarding claim 69, i. Performing a color characterization analysis of the 
template image is illustrated by Krumm in figure 2 by reference number 204. The color 
characterization analysis of the template image corresponds to the created model 
histograms associated with people or objects, ii. for each of the plurality of sample 
regions, performing a color characterization analysis for the sample region is illustrated 
by Krumm in figure 3 by reference number 202, iii. determining the measure of 
difference between the color information of each sample region and the color 
information of the template image comprises comparing information obtained in the 
color characterization analysis of the sample region with information obtained in the 
color characterization of the template image is illustrated by Krumm in figure 2 by 
reference number 204. The color histograms of the regions of the target image are 
compared with the color histograms of the template image; 

Regarding claim 70, i. the template image and the target image each comprising 
a plurality of pixels is explained by Krumm in column 11, lines 17-23, ii. Examining color 
information of at least a subset of pixels is explained by Krumm in column 10, lines 33- 
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37. Krumm explains that the color histogram is created for the extracted regions 
(corresponding to a subset of the color image pixels), iii. Assigning each examined pixel 
to a color category that corresponds to a portion of a color space is explained by Krumm 
in column 10, line 63 to column 1 1 , line 6. Each color space region defined by Krumm 
corresponds to a portion of the RGB color space, iv. Determining information indicative 
of the allocation of the examined pixels across color categories is explained by Krumm 
in column 1 1 , lines 17-23. The information indicative of the allocation of the examined 
pixels is the count of pixels belonging to each category, v. Comparing the information 
obtained in the color characterization analysis of the region to the information obtained 
in the color characterization analysis of the template image comprises comparing the 
allocation of the examined pixels across color categories for the sample region and the 
template image is explained by Krumm in column 1 1 , lines 49-64. 

13. Claim 71 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krumm 
'622 in view of Hsu et al, and further in view of Nelson et al. 

Regarding claim 71', i. The color characterization analysis performed for the 
template image further comprising determining one or more dominant color categories, 
wherein one or more dominant categories are assigned a relatively larger proportion of 
examined pixels, with respect to other categories of the color space is not explicitly 
explained by Krumm or Hsu et al. However, Nelson et al explain in column 1 1 , lines 54- 
64 that only the top Nth most significant bins are retained in the color histogram of an 
image, ii. Comparing information obtained in the color characterization analysis of the 
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sample region with information obtained in the color characterization analysis of the 
template image comprising comparing the dominant color categories of the sample 
region and the template region is not explicitly explained by Krumm or Hsu et al. 
However, Nelson et al explain in the abstract that characterizations of multimedia 
(including image) works are created in order to retrieve multimedia from a database. By 
determining and comparing the most dominant color categories of a template image, the 
amount of data being retained for each image would be reduced. Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to determine one or more dominant color categories for the template image from the 
color histogram created in the system of Krumm, and compare the dominant regions of 
the color characterization because the computational power and memory requirement 
for the image matching system would be reduced. 

14. Claim 72 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krumm 
'622 in view of Hsu et al, and further in view of Kato et al. (US 6,665,446). 

Regarding claim 72, i. Examining color information for each pixel in the template 
image is illustrated by Krumm in figure 2 by reference number 204. Each pixel of the 
template image is processing in order to create the "previously created model histogram 
associated with people or objects", ii. Examining color information of only a subset of 
the pixels in a region of the target image is not explicitly explained by Krumm or Hsu et 
al. However, Kato et al illustrate a sub-sampling size of four in figure 3. The sub- 
sampling method of Kato et al can be used in the system of Krumm and Hsu et al in 



Application/Control Number: 09/639,420 Page 22 

Art Unit: 2621 

order to reduce the processing required to determine the color characterization of each 
region of the target image. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to examine color information of only a 
subset of the pixels in a region of the target image because the information in each 
region of the target image in the system of Krumm and Hsu et al would be reduced, 
reducing the computational power and memory required. 

15. Claim 73 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krumm 
'622 in view of Hsu et al, and further in view of Chen et al. (1449, Item B7). 

Regarding claim73, i. The template image and the target image comprising hue, 
saturation, and intensity (HSI) color information is explained by Krumm in column 10, 
lines 38-49. Krumm explicitly explains that the color images may contain HIS color 
information, ii. Examining HSI information of least a subset of pixels is explained by 
Krumm in column 10, lines 38-49. Krumm explicitly explains that the HIS information 
may be examined for the template image and the segments of the target image, 
iii. Assigning each examined pixel to a color category that corresponds to a portion of 
HSI color space is not explicitly explained by Krumm or Hsu et al. Krumm does explain 
color categories corresponding to the RGB color space in column 10, line 63 to column 
1 1 , line 6. Krumm further explains that the HSI color characterization may be used 
instead of the RGB characterization in column 10, lines 44-49. However, Chen et al 
explain that color categories are created according to the HSI color space on page 644, 
paragraphs 5 and 6 (Q2 and Q3). Chen et al explain that the pixels of an image are 
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assigned to the color categories on page 644, paragraph 2. Chen et al explain that 
color categories are assigned to portions of the HSI color space in order to reduce the 
number of histogram color bins, thereby reducing the processing required to determine 
the degree of similarity between the template image and the target image regions. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to assign each examined pixel to a color category that corresponds 
to a portion of HSI color space, as explained by Chen et al, in the system of Krumm and 
Hsu et al because Krumm suggests the HSI color space as an alternative to the RGB 
color space and the color categories reduce the processing required in the system of 
Krumm and Hsu et al. 

16. Claims 74-76 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Krumm '622 in view of Hsu et al, and further in view of Park et al. 

Regarding claim 74, i. Determining a first step size to use in the performing the 
first-pass search through the target image is not explicitly explained by Krumm or Hsu et 
al. Hsu et al do illustrate a coarse search in figure 3 by reference number 304, but do 
not explicitly explain that a first step size is used. However, Park et al illustrate a first 
step size of two pixels in figure 1 by the two vectors originating from coordinate (0,0), 
ii. Determining a second step size, wherein the second step size is smaller than the first 
step size is not explicitly explained by Krumm or Hsu et al. Hsu et al do illustrate a fine 
search in figure 3 by reference number 306, but do not explicitly explain that a second 
step size is used. However, Park et al illustrate a second step size of one pixel in figure 
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1 by the two vectors originating from (4, -6) and (-6, 6). The second step size is one, 
which is smaller than the first step size of two, iii. Searching the proximal regions 
proximal to the color match candidate areas using the second step size is not explicitly 
explained by Krumm or Hsu et al. However, Park et al illustrate in figure 1 that the 
proximal regions to coordinates (4, -6) and (-6, 6) are searched using the second step 
size. Park et al explain in column 1, lines 11-18 that the multiple step search sizes 
reduce the calculations required to determine the best match in a target image. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to search the proximal regions proximal to the color match 
candidate areas using the second step size in order to determine matches between a 
plurality of regions within the target image and a template image in the system of 
Krumm because the region selection process would be more efficient; 

Regarding claim 75, i. Determining a plurality of sample regions in the proximal 
region at which to sample the color information of the target image is not explicitly 
explained by Krumm or Hsu et al. Krumm does illustrate that color information is 
sampled at every target region in figure 2 by reference number 202. However, Park et 
al illustrate eight sample regions proximal to the coordinate (0,0) in figure 1 , ii. 
Determining a measure of difference between the color information of the sample region 
and the color information of the template image for each of the plurality of sample 
regions in the proximal region is not explicitly explained by Krum or Hsu et al. The 
system of Krumm uses color information in order to determine the difference between a 
template image and regions of a target image as illustrated in figure 2 by reference 
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number 204. However, Park et al explain in column 1, lines 49-52 that a difference 
measure is determined at each of the proximal search regions, iii. The best-matching 
region in the proximal region being a sample region with a smallest measure of 
difference is not explicitly explained by Krumm or Hsu et al. However, Park et al explain 
in column 1, lines 49-52 that the minimum block distance measure is determined from 
one of the center search location and eight proximal search locations. Park et al explain 
in column 1, lines 11-18 that the four-step search reduces the calculations required to 
determine the best match in a target image. Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to determine a plurality 
of sample regions at which to sample color information, determine a best-matching 
region in the proximal region with the smallest measure of difference between the color 
information so that the color matching system of Krumm and Hsu et al would be more 
efficient. 

Regarding claim 76, i. For each best-matching region found, determining a 
measure of difference between the color information of the region and the color 
information of the template image is illustrated by Krumm in figure 2 by reference 
number 204, ii. Designating at least a subset of the best-matching regions as the final 
match regions is not explicitly explained by Krumm or Hsu et al. However, Park et al 
illustrate in figure 1 that the best match regions correspond to the regions at coordinates 
(-5, 7) and (3, -7). By designating only the best search regions in the final step as the 
final match regions, less computational power and memory would be required to store 
and process the search regions. Therefore, it would have been obvious to one of 
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ordinary skill in the art at the time the invention was made to Designate at least a subset 
of the best-matching regions as the final match regions in the system of Krumm so that 
the computational power and memory required in the system of Krumm would be 
reduced, iii. The each best matching region designated as a final match region, the 
measure of difference between the color information of the final match region and the 
color information of the template image being less than a threshold is illustrated by 
Krumm in figure 2, by reference number 206. 

17. Claim 85 rejected under 35 U.S.C. 103(a) as being unpatentable over Krumm 
(6,532,301 ) in view of Chen et al (1449, Item B7). 

Regarding 85, Krumm '301 teahces number of limitation of the claim however, 
determining contributions which a pixel should make to a plurality of color categories 
and distributing the weight of the pixel across the plurality of color categories in 
accordance with the determined contributions is not explicitly explained by Krumm. 
Chen et al illustrate overlapping membership functions for soft-decision histogramming 
in figure 1b. The weight of the pixel is distributed due to the overlapping of the 
membership functions, wherein a value for "c" (on the x-axis) corresponds to multiple 
color categories. The values of the membership functions at "c" determine the weights 
of the pixel in each category. Chen et al explain on page 644, paragraph 7, that soft- 
decision histogramming is an improvement over hard-decision histogramming, as used 
in the system of Krumm, because small, random color variations will not render different 
histograms. Therefore, it would have been obvious to one of ordinary skill in the art at 
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the time the invention was made to use soft-decision histogramming, as suggested by 
Chen et al, over hard-decision histogramming in the system of Krumm because soft- 
decision histogramming provides more accurate and consistent representations of 
images exhibiting small, random color variations. 

18. Claims 63 and 84 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Krumm (6,532,301 ) in view of Nelson et al. 

Regarding claim 63 which is representative of claim 84, Krumm '310 teahces 
number of limitation of the claims however, Krumm fails to specifically teach said 
comparing information obtained in the color characterization analysis of each region of 
the target image to information obtained in the color characterization analysis of the 
template image comprises: for each color category of the color space, comparing the 
percentage of template image pixels assigned to the color category to the percentage of 
target image region pixels assigned to the color category. Nelson et al explain 
determining the dominant color categories in column 1 1 , lines 54-64. Nelson et al also 
illustrate filtering and normalizing the tokens (i.e. categories) in figure 6 by reference 
numbers 630. The normalization of the categories determines the percentage of pixels 
present in each color category rather than the number of pixels. Nelson et al explain in 
the abstract that characterizations of multimedia (including image) works are created in 
order to retrieve multimedia from a database. Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to compare the 
percentage of template image pixels assigned to the dominant color category to the 



Application/Control Number: 09/639,420 Page 28 

Art Unit: 2621 

percentage of target image region pixels assigned to that color category because 
comparing the percentages of only the dominant categories reduces the computational 
power and memory requirement in the system of Krumm. 

Allowable Subject Matter 

19. Claims 59-61 and 83 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. The following is a statement of 
reasons for the indication of allowable subject matter: The prior art of record specifically 
Krumm 6,532,301 does not teach the template and the target image comprise hue, 
saturation and intensity (HIS) color information, examining the HIS information of the 
subset of pixels and assigning each examined pixel to a color category that corresponds 
to a portion of a color space comprises assigning each examined pixel to a color 
category that corresponds to a portion of HIS color space and searching the target 
image using a template image for matching of claims 59 and 83 combined with other 
features and elements of the claims . 
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